Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code changes for the implementation of the low-resolution B #1441

Merged
merged 27 commits into from
Feb 4, 2025

Conversation

guillaumevernieres
Copy link
Contributor

@guillaumevernieres guillaumevernieres commented Jan 13, 2025

As the title says.

Dependencies

Work done in this PR

Interp to low-resolution:

  • variance partitioning
  • preparation of the perturbations for the envar
  • hybrid weights
  • ...

Testing

Visual check showing the variance explained by the steric height (30 members)
steic-explained-variance
left is native resolution, right is on the 1/2 deg grid.

"Science" note: The explained variance above is almost the opposite to what I would have expected, and what we saw in the previous offline ensemble, or even what we see in tendencies. Check if there's a bug in the explained variance calculation.

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmaerosnowDA
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@guillaumevernieres guillaumevernieres self-assigned this Jan 13, 2025
@guillaumevernieres guillaumevernieres added the hera-GW-RT Queue for automated testing with global-workflow on Hera label Jan 13, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Jan 13, 2025
@emcbot
Copy link

emcbot commented Jan 13, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Mon Jan 13 21:38:37 UTC 2025 on hfe10
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon Jan 13 22:32:01 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Mon Jan 13 22:55:24 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 53

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jan 13, 2025
shlyaeva
shlyaeva previously approved these changes Jan 14, 2025
Copy link
Collaborator

@shlyaeva shlyaeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not test, but I reviewed all the code changes, they look good to me.

@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Passed Automated testing with global-workflow successful on Hera labels Jan 15, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Jan 15, 2025
@emcbot
Copy link

emcbot commented Jan 15, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Wed Jan 15 21:37:00 UTC 2025 on hfe09
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Wed Jan 15 22:26:38 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Wed Jan 15 22:47:43 UTC 2025
Tests: 92% tests passed, 4 tests failed out of 53
	2042 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	2043 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
	2054 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlchkpt_202103250000 (Failed)
	2055 - test_gdasapp_C48mx500_hybAOWCDA_gdas_marineanlfinal_202103250000 (Failed)
Tests: see output at /scratch1/NCEPDEV/da/role.jedipara/CI/GDASApp/workflow/PR/1441/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hera-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Jan 15, 2025
@guillaumevernieres guillaumevernieres added hercules-GW-RT Queue for automated testing with global-workflow on Hercules and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Jan 16, 2025
@guillaumevernieres guillaumevernieres added hera-GW-RT Queue for automated testing with global-workflow on Hera and removed hera-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Feb 3, 2025
@emcbot emcbot added hera-GW-RT-Running Automated testing with global-workflow running on Hera and removed hera-GW-RT Queue for automated testing with global-workflow on Hera labels Feb 3, 2025
@emcbot
Copy link

emcbot commented Feb 4, 2025

Automated GW-GDASApp Testing Results:
Machine: hera

Start: Mon Feb  3 23:46:06 UTC 2025 on hfe07
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb  4 00:34:26 UTC 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Tue Feb  4 01:39:01 UTC 2025
Tests: 100% tests passed, 0 tests failed out of 54

@emcbot emcbot added hera-GW-RT-Passed Automated testing with global-workflow successful on Hera and removed hera-GW-RT-Running Automated testing with global-workflow running on Hera labels Feb 4, 2025
@emcbot
Copy link

emcbot commented Feb 4, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Mon Feb  3 12:19:57 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Mon Feb  3 13:00:49 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Mon Feb  3 19:55:29 CST 2025
Tests: 98% tests passed, 1 tests failed out of 54
	2039 - test_gdasapp_C48mx500_3DVarAOWCDA_gdas_waveinit_202103241800 (Timeout)
Tests: see output at /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1441/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 4, 2025
@RussTreadon-NOAA RussTreadon-NOAA added hercules-GW-RT Queue for automated testing with global-workflow on Hercules orion-GW-RT Queue for automated testing with global-workflow on Orion and removed hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera labels Feb 4, 2025
@guillaumevernieres
Copy link
Contributor Author

Weird, it looks like it used the wrong commit of the g-w branch. I merged develop before resubmitting the ci but that;s not what was cloned here: /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1441/global-workflow

@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules orion-GW-RT-Running Automated testing with global-workflow running on Orion and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules orion-GW-RT Queue for automated testing with global-workflow on Orion labels Feb 4, 2025
@emcbot
Copy link

emcbot commented Feb 4, 2025

Automated GW-GDASApp Testing Results:
Machine: orion

Start: Tue Feb  4 08:04:59 AM CST 2025 on orion-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb  4 09:11:58 AM CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Tue Feb  4 09:39:22 AM CST 2025
Tests: 100% tests passed, 0 tests failed out of 54

@emcbot emcbot added orion-GW-RT-Passed Automated testing with global-workflow successful on Orion and removed orion-GW-RT-Running Automated testing with global-workflow running on Orion labels Feb 4, 2025
@emcbot
Copy link

emcbot commented Feb 4, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Tue Feb  4 08:05:32 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb  4 08:46:00 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Tue Feb  4 10:09:04 CST 2025
Tests: 100% tests passed, 0 tests failed out of 54

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 4, 2025
@RussTreadon-NOAA RussTreadon-NOAA self-requested a review February 4, 2025 16:10
Copy link
Contributor

@RussTreadon-NOAA RussTreadon-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GDASApp ctests including C48mx500_3DVarAOWCDA and C48mx500_hybAOWCDA pass on Hera, Hercules, and Orion.

Approve.

Copy link
Collaborator

@AndrewEichmann-NOAA AndrewEichmann-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok

@guillaumevernieres guillaumevernieres merged commit fa09326 into develop Feb 4, 2025
35 checks passed
@guillaumevernieres guillaumevernieres deleted the feature/lowres-b branch February 4, 2025 16:46
@shlyaeva
Copy link
Collaborator

shlyaeva commented Feb 4, 2025

Here are the changes that I needed to do for the recenter task to succeed with a zero exit status:

diff --git a/parm/soca/berror/soca_ensrecenter.yaml b/parm/soca/berror/soca_ensrecenter.yaml
index b700601..b7d8e20 100644
--- a/parm/soca/berror/soca_ensrecenter.yaml
+++ b/parm/soca/berror/soca_ensrecenter.yaml
@@ -60,5 +60,5 @@ output increment:
   date: '{{ MARINE_WINDOW_END_ISO }}'
   exp: trash
   type: incr
-  output file: 'ocn.recenter.incr.%mem%.nc'
+  output file: 'recenter.incr.%mem%.nc'
   pattern: '%mem%'

and

diff --git a/utils/soca/gdas_ens_handler.h b/utils/soca/gdas_ens_handler.h
index 741055c..dd5fe9c 100644
--- a/utils/soca/gdas_ens_handler.h
+++ b/utils/soca/gdas_ens_handler.h
@@ -169,7 +171,30 @@ namespace gdasapp {
           postProcIncr.setToZero(incr);

           // Save the increments used to initialize the ensemble forecast
-          result = postProcIncr.save(mom6_incr, i+1);
+          result = postProcIncr.save(mom6_incr, i+1, {"ocn"});

It fixes two issues: missing ice file (that we don't use in this usecase anyway) and wrong ocean file name.
I didn't test beyond this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hera-GW-RT-Passed Automated testing with global-workflow successful on Hera hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules orion-GW-RT-Passed Automated testing with global-workflow successful on Orion
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants